Multi-objective Model Predictive Optimization using Computational Intelligence

In many engineering design problems, the explicit function form of objectives/constraints can not be given in terms of design variables. Given the value of design variables, under this circumstance, the value of those functions is obtained by some simulation analysis or experiments, which are often expensive in practice. In order to make the number of analyses as few as possible, techniques for model predictive optimization (also referred to as sequential approximate optimization or metamodeling) which make optimization in parallel with model prediction have been developed. In this paper, we discuss several methods using computational intelligence for this purpose along with applications to multi-objective optimization under static/dynamic environment. 1 Brief Review of Model Predictive Methods To begin with, we shall review several typical methods for model prediction. Response Surface Method (RSM) has been probably most widely applied to our aim [6]. The role of RSM is to predict the response y for the vector of design variables x = (x1, . . . , xn) on the basis of the given sampled observations (x̃i, ỹi), i = 1, . . . , l. Usually, Response Surface Method is a generic name, and it covers a wide range of methods. Above all, methods using design of experiments are famous. However, many of them use relatively low order (say, 1st or 2nd) polynomials on the basis of statistical analysis in design variable space. Theymay provide a good approximation Hirotaka Nakayama Department of Info. Sci. & Sys. Eng., Konan University, Kobe 658-8501, Japan, e-mail: [email protected] Yeboon Yun Department of Reliability-based Information Systems Engineering, Kagawa University,Takamatsu 761-0396, Japan, e-mail: [email protected] Hirotaka Nakayama and Yeboon Yun of black-box functions with a mild nonlinearity. It is clear, however, that in cases in which the black-box function is highly nonlinear, we can obtain better performance by methods using computational intelligence such as RBFN (Radial Basis Function Networks) or SVR (Support Vector Regression) taking into account not only the statistical property in design variable space but also that of range space of the blackbox function (in other words, the shape of function). In design of experiments, for example, D-optimality may be used for selecting a new additional sample to minimize the variance covariance matrix of the least squared error prediction. With the design matrix X , this reduces to minimize the matrix (X X)−1 which is attained by maximizing det(X X). This is the idea of D-optimality in design of experiments. Other criteria are possible: to minimize the trace of (X X)−1 (A-optimality), to minimize the maximal value of the diagonal components of (X X)−1 (minimax criterion), to maximize the minimal eigen value ofXT X (E-optimality). In general, D-optimality criterion is widely used for many practical problems. Jones et al. [5] suggested a method called EGO (Efficient Global Optimization) for black-box objective functions. They applied a stochastic process model for predictor and the expected improvement as a figure of merit for additional sample points. Regard y as a realized value of the stochastic variable Y , and let f min be the minimal value of p-samples which are evaluated already. For minimization cases, the improvement at x is I = max ( f min − Y, 0 ) . Therefore, the expected improvement is given by E[I(x)] = E [ max ( f min − Y, 0 )] . We select a new sample point which maximizes the expected improvement. Although Jones et al. proposed a method for maximizing the expected improvement by using the branch and bound method, we can select the best one among several candidates which are generated randomly in the design variable space. It has been observed through our experiences that this method is time consuming. 2 Using Computational Intelligence Recently, the authors proposed to apply machine learning techniques such as RBF (Radial Basis Function) networks and Support Vector Machines (SVM) for approximating the black-box function [7], [8]. There, additional sample points are selected by considering both global and local information of the black-box function. Support vector machine (SVM) has been recognized as a powerful machine learning technique. SVM was originally developed for pattern classification and later extended to regression ([1], [13]). In pattern classification problems with two class sets, it generalizes linear classifiers into high dimensional feature spaces through nonlinear mappings defined implicitly by kernels in the Hilbert space so that it may produce nonlinear classifiers in the original data space. Linear classifiers then are optimized to give the maximal margin separation between the classes. 320 Multi-objective Model Predictive Optimization using Computational Intelligence This task is performed by solving some type of mathematical programming such as quadratic programming (QP) or linear programming (LP). Linear classifiers on the basis of goal programming, on the other hand, were developed extensively in 1980’s [3], [4]. The authors developed several varieties of SVM using multi-objective programming and goal programming (MOP/GP) techniques [10]. In the goal programming approach to linear classifiers, we consider two kinds of deviations: One is the exterior deviation ξi which is a deviation from the hyperplane of a point xi improperly classified; The other one is the interior deviation ηi which is a deviation from the hyperplane of a point xi properly classified. Several kinds of objective functions are possible in this approach as follows: i) minimize the maximum exterior deviation (decrease errors as much as possible), ii) maximize the minimum interior deviation (i.e., maximize the margin), iii) maximize the weighted sum of interior deviation, iv) minimize the weighted sum of exterior deviation. Introducing the objective iv) above leads to the soft margin SVM with slack variables (or, exterior deviations) ξi (i = 1, . . . , l) which allow classification errors to some extent. Taking into account the objectives (ii) and (iv), we can have the same formulation of ν-support vector algorithm developed by Schölkopf et al. [12]. Although many variants are possible, μ−ν−SVM considering the objectives i) and ii) is promising, because μ−ν−SVM for regression has been observed to provide a good sparse approximation [10]. The primal formulation of μ−ν−SVR is given by minimize w,b,ε,ξ,ξ́ 1 2 ∥w∥22 + νε + μ(ξ + ξ́) subject to ( w zi + b ) − yi ! ε + ξ, i = 1, . . . , l, yi − ( w zi + b ) ! ε + ξ́, i = 1, . . . , l, ε, ξ, ξ́ " 0, where ν and μ are trade-off parameters between the norm of w and ε and ξ (ξ́). Applying the Lagrange duality theory, we obtain the following dual formulation of μ−ν−SVR: 321 Hirotaka Nakayama and Yeboon Yun maximize αi,άi − 1 2 l